feat: backport batch chonk verifier with split peer/RPC architecture to v4#21638
Open
feat: backport batch chonk verifier with split peer/RPC architecture to v4#21638
Conversation
…to v4 Adds batch IPA verification for Chonk proofs with a split verifier architecture: - peerChonkVerifier (BatchChonkVerifier, batch=8) for P2P gossip/reqresp - rpcChonkVerifier (QueuedIVCVerifier wrapping BatchChonkVerifier batch=1) for RPC C++ changes: - Add IPA::batch_reduce_verify for single-MSM batch IPA verification - Refactor IPA::reduce_verify_internal_native to use read_transcript_data - Add ChonkVerifier::reduce_to_ipa_claim for non-IPA verification extraction - Add ChonkBatchVerifier with parallel reduce + batch IPA + bisection - Add ChonkBatchVerifierService with FIFO result streaming - Add batch verifier RPC commands (Start/Queue/Stop) to bbapi TypeScript changes: - Add BatchChonkVerifier class using bb batch verifier service via FIFO - Split single proofVerifier into peerChonkVerifier and rpcChonkVerifier - Make P2P proofVerifier optional (skip proof validation in fake mode) - Add BB_RPC_VERIFY_BATCH_SIZE and BB_PEER_VERIFY_BATCH_SIZE config - Simplify QueuedIVCVerifier constructor to (verifier, concurrency)
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backports the batch chonk verifier from #21460 to v4, with the split peer/RPC verifier architecture.
Architecture
Two verifier instances per node:
BatchChonkVerifier, batch_size=8) — for P2P gossip and reqresp tx validationQueuedIVCVerifierwrappingBatchChonkVerifierbatch_size=1) — for RPCsendTxvalidation with concurrency controlC++ (barretenberg)
IPA::batch_reduce_verify— batch-verifies multiple IPA proofs with a single SRS MSM using random alpha-batchingIPA::read_transcript_data— refactored out ofreduce_verify_internal_nativefor reuse by batch verificationChonkVerifier::reduce_to_ipa_claim— runs all non-IPA verification (MegaZK, databus, Goblin), returns IPA claimChonkBatchVerifier— coordinator with parallel reduce (work-stealing), batch IPA check, and bisection for failure isolationChonkBatchVerifierService— wrapsChonkBatchVerifierwith FIFO result streaming via named pipeChonkBatchVerifierStart,ChonkBatchVerifierQueue,ChonkBatchVerifierStopTypeScript (yarn-project)
BatchChonkVerifier— persistent bb native process, sends proofs via msgpack RPC, receives results via named FIFOproofVerifierintopeerChonkVerifierandrpcChonkVerifierinAztecNodeServiceproofVerifieroptional — skip proof validation when running in fake modeQueuedIVCVerifiersimplified constructor:(verifier, concurrency)instead of(config, verifier)BB_RPC_VERIFY_BATCH_SIZE,BB_PEER_VERIFY_BATCH_SIZE(default fromBB_NUM_IVC_VERIFIERSfor backwards compat)Test plan
chonk_batch_verifier.test.cpp(batch of 2, flush on shutdown, tampered proof bisection, invalid VK index)bbbinary builds successfully with all new commands